Nonparametric Unsupervised Classification

Unsupervised classification methods learn a discriminative classifier from unlabeled data, which has been proven to be an effective way of simultaneously clustering the data and training a classifier from the data. Various unsupervised classification methods obtain appealing results by the classifiers learned in an unsupervised manner. However, existing methods do not consider the misclassification error of the unsupervised classifiers except unsupervised SVM, so the performance of the unsupervised classifiers is not fully evaluated. In this work, we study the misclassification error of two popular classifiers, i.e. the nearest neighbor classifier (NN) and the plug-in classifier, in the setting of unsupervised classification. The upper bound for the misclassification error of both classifiers involves only pairwise similarity between the data points. We prove that the error of the plug-in classifier is asymptotically bounded by the weighted volume of cluster boundary [1]. Also, the normalized graph Laplacian from the induced similarity kernel recovers different types of transition kernels for Diffusion maps [2,3], which reveals the close relationship between manifold learning and unsupervised classification. We show that with the normalized graph Laplacian, the similarity kernel induced by the misclassification error of the plug-in classifier corresponds to the Fokker-Planck operator; and the similarity kernel induced by the volume of misclassified region by the plug-in classifier correspond to the Laplace-Beltrami operator on the data manifold.